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ABSTRACT 

A part of NASA's Space Station will be a 
Flight Telerobotic Servicer (FTS) used to help 
assemble, service, and maintain the Space 
Station. Since the human operator will be 
required to control the FTS the design of the 
human-telerobot interface must be optimized 
from a Human Factors perspective. 

Simulation has been used as an aid in the 
development of complex systems. Simulation has 
been especially useful when it has been applied 
to the development of complex systems. 

Simulation should ensure that the hardware and 
software components of the human-telerobot 
interface have been designed and selected so 
that the operator's capabilities and limitations 
have been accommodated for since this is a 
complex system where few direct comparisons 
to existent systems can be made. Three broad 
areas of the human-telerobot interface where 
simulation can be of assistance will be 
described. The use of simulation can not only 
result in a well-designed human-telerobot 
interface, but it can also be used to ensure that 
components have been selected to best meet 
system's goals and for operator training. 

INTRODUCTION 

The Space Station is a NASA project which, 
when completed in the mid-1 990's, will function 
as a permanently manned orbiting space 
laboratory. A part of the Space Station will be a 
remotely controlled Flight Telerobotic Servicer 
(FTS). The FTS, a project led by NASA's Goddard 
Space Flight Center, will be used to help 


assemble, service, and maintain the Space 
Station and various satellites. The use of the 
FTS will help ensure the safety and productivity 
of space-based tasks normally accomplished by 
astronauts performing outside the pressurized 
spacecraft. For the short-term, control of the 
FTS will be dependent primarily on the human 
operator. Since the human operator will be a 
part of the telerobotic system, then it is 
important that the human-telerobot interface be 
well-designed from a Human Factors 
perspective. It is critical that the components 
of this interface be designed so that the human 
operator's capabilities and limitations are best 
accommodated for within the structure of 
specific task requirements. To emphasize the 
importance of a well-designed human-telerobot 
interface, one study found that simply the 
selection of an appropriate control device, based 
upon the operator's capabilities and the 
requirements of the task, can more than double 
the productivity of the telerobotic system 
(O'Hara, 1986). 

With the system development process 
becoming more complex and expensive, more 
emphasis is being placed on the evaluation of 
systems during early stages of the development 
cycle. The design of systems that include human 
operators is especially complex because 
determining overall systems performance is 
dependent upon the interaction of the human 
operator, hardware components, and software 
components (Chubb, Laughery, and Pritsher, 
1987). Adequately evaluating the performance 
of a system during the design cycle is becoming 
increasing more difficult when using the static 
evaluation tools traditionally available to the 
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Human Factors Engineer, such as job and task 
analysis (Geer, 1981). It is becoming more 
common for systems developers to use 
simulation as a design tool instead of hardware 
models (Gawron and Polito, 1985) and for Human 
Factors Engineers to use simulation to enhance 
the use of static evaluation tools. This is 
because more sophisticated analysis tools are 
needed that will allow a controlled evaluation of 
the human operator/hardware 
components/software components interaction 
(Chubb, Laughery, and Pritsher, 1987). 

This paper will cover the various uses of 
simulation, the elements of the human-telerobot 
interface, and how simulating the 
human-telerobot interface on the Space Station 
will result in a better designed system. Before 
focusing the discussion specifically to the 
simulation of the human-telerobot interface, it 
will be useful to briefly define simulation and 
to cover the major uses of system simulation -- 
independent of the type of system that is being 
simulated. There will then be a discussion of 
the areas of the human-telerobot interface and 
how simulation can contribute to a better 
designed user interface from a Human Factors 
perspective. 

USES OF SIMULATION 

Simulation is the process of imitating or 
duplicating the actions or processes of some 
system in a controlled environment (Arya, 

1985). Emphasis should be placed on the word 
"controlled." System simulation, either 
hardware, computer, or a combination of the 
two, has been used for decades. This paper will 
describe four major uses of simulation. One use 
of simulation is to study the effectiveness of 
various hardware/software components on 
overall system's performance. The advantages 
of using simulation within this context are cost 
- it is cheaper to simulate a system than it is 
to build one; time - simulating a system is 
usually faster than building it; feasibility - 
because of the size and complexity of some 
systems, it is not possible to evaluate them in 
the real world, therefore, simulation serves the 
function of systems verification; safety- some 
systems operate in dangerous environments and 
can only be evaluated safely with the use of 


simulation; and prediction - with the use of 
simulation, a system's performance and 
processes can be speeded up so that future 
behavior can be predicted (Arya, 1985). 

A second use of simulation is to determine 
the effects of various hardware/software 
components on simulated human performance. 
This approach utilizes mathematical models of 
human performance to assist the simulation 
process. In this, as well as the approach 
mentioned above, man-in-the-loop is not a part 
of the evaluation. 

A third use of simulation is to investigate 
the effects of various hardware/software 
components on actual human performance. This 
approach can be taken in an attempt to match 
systems components and operator capabilities 
and limitations in order to ensure optimal 
systems and operator performance. This 
approach can be taken in an attempt to add 
greater fidelity, and thus, external validity to 
the data that are gathered in the analysis. 

The last use of simulation to be addressed in 
this paper is to train operators to eventually use 
a real-world system. The major benefits of 
simulation as a training aid are in the areas of 
scheduling - training is not affected by 
weather or the need to perform operational 
missions; cost - simulator training is 
significantly less expensive than prime system 
training; safety - reduces the exposure of 
operators and the prime system to the hazards 
of the operating environment; control of training 
conditions - control of environmental and 
human interaction conditions that may be a part 
of the operating environment; learning 
enhancement - system malfunctions and 
environmental conditions can be included in the 
training; and performance enhancement - 
inclusion of critical missions that are 
difficult to train for in the real world (Flexman 
and Stark, 1987). 

As the above list indicates, simulation has 
significant usage as an aid in the development of 
systems. It can have even greater significance 
in the design and development of novel systems 
- systems that have never existed before and 
where few direct comparisons to existent 
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systems can be made. The human-telerobot 
system that will be used on the Space Station is 
such a novel system. 

Even though industrial robots and 
teleoperators are heavily used in such areas as 
the nuclear industry and in underwater 
activities, there are major differences between 
these applications and the telerobot system to 
be used on the Space Station -- one of these 
being the zero-gravity factor. There is also a 
limited number of direct comparisons which can 
be made from the Remote Manipulator System 
(RMS) used on the Space Shuttle and the 
proposed telerobot system. The review of the 
literature concerning these systems has 
provided answers to some important design 
issues, but there are major limitations to how 
far these data can be generalized to the 
human-telerobot interface on the Space Station. 

It is thus proposed that the use of simulation 
in the design and development of the 
human-telerobot interface on the Space Station 
will be very beneficial. Simulation should serve 
as an aid in the selection and design of hardware 
and software components to ensure maximum, 
error-free performance. Simulation should be 
worthwhile especially for its ability to 
simulate the effects of zero-gravity on 
performance. This can be accompished in 
underwater conditions (e. g., Weightless 
Environment Test Facility at Johnson Space 
Center) and when people are exposed to 
momentary weightless conditions which can 
occur in certain aircraft (e. g., KC-135). 

Operator performance at manipulation tasks 
while in a one-gravity environment may well not 
be generalizable to weightless states. 

Simulation of the interface should also have the 
benefit of helping engineers to detect flaws in 
the design of components of the interface which 
would adversely affect system and/or operator 
performance. It is obviously important that any 
mistakes of this type be detected early and far 
before the design is finalized or manufacture of 
the system has occurred. 

INFORMATION NEEDS OF THE OPERATOR 

There are three broad areas of the 
human-telerobot interface where simulation can 


be of assistance: operator information needs, 
control devices, and workstation layout. These 
three areas and examples of various components 
are listed in Table 1 . The information needs of 
the operator will vary depending upon the tasks 
to be performed. The operator will need 
information concerning the location and 
orientation of the telerobot in space, the health 
status of the telerobot, visual feedback from the 
viewing system, the status of any 
transportation devices, the status of the 
workpiece, and the status of the hardware in the 
control workstation. 

TABLE 1 

Three areas of the human-telerobot interface 
and example components 


1 . Information needs of the operator 

• Location of telerobot 

• Status of transportation devices 

• Status of workpiece 

• Status of workstation 

• Force feedback 

• Visual feedback 

Camera position and number 
Spatial orientation of image 
Monitor type, placement, and number 
Illumination 

2. Control devices considered 

• Miniature master controllers 

• 3 or 6 degree-of-freedom hand controllers 

• Exoskeleton controllers 

• Head-slaved controllers 

• Dedicated switches 

• Programmable display pushbuttons 

• Voice commanded systems 

• Computers 

3. Telerobot workstation 

• Hardware layout 

• Software layout 

• Restraint systems 


Regarding visual feedback, the visual system 
may well be the single most important source of 
information for the operator (Smith and Stuart, 
in press). Some of the issues related to the 
visual system are concerned with camera 
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position and number, the spatial-orientation of 
the image presented to the operator, and monitor 
type, placement, and number. For example, when 
performing a remote manipulation task in real 
time, the operator can view the remote scene 
either by looking through a window, or with the 
use of cameras. For most of the tasks that will 
be performed in space, a direct view of the 
working area will either not be available, or 
will not provide the necessary visual cues for 
teleoperation. Therefore, cameras will provide 
the primary mode of feedback to the operator 
concerning manipulator position, orientation, 
and rate of movement. Operators normally use 
the body of the manipulator as a reference point 
when making control inputs, but, if the Space 
Station's external cameras are placed such that 
the camera view is not normal to the 
manipulator (normal refers to placement behind 
the shoulder of the arm) then the visual 
feedback will be spatially displaced. Spatial 
displacement is an unfortunate consequence of 
attempts to provide complete visual information 
to the operator; however, when the camera 
placement is not normal it should be avoided if 
at all possible or compensation techniques 
should be employed such as referencing the 
control device to the perceived motion on a 
monitor. 

Spatially displaced feedback can take on 
different forms: angular displacement - view is 
displaced horizontally within the transverse 
plane or vertically within the median plane; 
reversal- view is facing the arm; 
inversion-reversal - view is upside down and is 
facing the arm; and inversion - view is upside 
down with respect to the manipulator arm. The 
image can also be temporally displaced - there 
are time delays in the visual feedback, as well 
as size distorted - the image is enlarged or 
reduced from its actual size. 

Spatial displacements adversely affect 
operator performance to varying degrees. 
Generally, they take on progressively more 
disturbance with angular displacement being the 
least disruptive and inversion displacement 
being the most disruptive. Temporal 
displacement interrupts the intrinsic temporal 
patterning of motion and causes severe 
disruptions in behavior. Much effort should be 


extended to prevent its occurrence. Size 
distortions generally do not affect performance 
to a great extent (Smith and Smith, 1962). 

Other visual system issues include how an 
operator will use multiple views of the task 
area and how operators can best use 
non-stereoscopic cues to depth perception. (The 
issue of stereo versus monocular viewing is 
still being discussed in the literature and will 
not be addressed in this paper.) Simulation of 
various task scenarios with human operators 
working within various hardware and software 
mockups, including sophisticated scene 
generation techniques, can serve as an aid in 
determining what types of information are 
needed and what types of information 
presentation enhancements should be used at 
various points within the sequence of task 
performance. An example of an information 
enhancement technique that simulation can 
investigate is the use of real-time moving 
graphics displays designed to help operators 
maintain their orientation while performing 
under potentially visually disorienting 
conditions. Other screen-viewing techniques 
should be investigated with the use of 
simulation in an attempt to avoid operator 
disorientation while performing manipulation 
tasks. 

CONTROL DEVICES 

Control devices will be used to control such 
things as telerobot activation, position, 
manipulators, end effectors, rate of movement, 
and the viewing system. Control devices being 
considered include manipulator controllers such 
as miniature master controllers with direct 
position control, 3 or 6 degree-of-freedom hand 
controllers using rate or force inputs, 
exoskeleton controllers using various position 
sensors to detect human arm configurations, 
head-slaved control, dedicated switches, 
programmable display pushbuttons, voice 
commanded systems, and computer displays 
with cursor-control devices which allow menu 
selections. Control device selection is 
important because it affects operator 
performance, workload, and preference. 
Computer simulated scenarios could be linked to 
the use of actual controllers to determine their 
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effects on operator performance across 
different manipulation tasks. The study of 
different controllers while performing various 
tasks could also be used as a means of 
determining whether the use of specific 
controllers is more muscle-fatigue inducing. 

WORKSTATION DESIGN 

The telerobot workstation consists of 
hardware elements, their interfaces, and the 
software that will allow the hardware to be 
used. The workstation is the point where the 
information and control inputs are made 
available to the operator. Just as with the 
selection of control devices, the workstation 
should be logically and functionally laid out to 
optimize operator performance and preference 
while minimizing workload and error rates. 
Simulation can help to determine optimal 
workstation layouts. A simple means of 
simulating the workstation layout is through the 
use of computer prototyping, but it is 
recommended that large-scale simulation be 
used as a means of designing and evaluating the 
telerobot workstation. 

CONCLUSIONS 

Many issues remain unresolved concerning 
the components of the human-telerobot 
interface mentioned above. It is then critical 
that these components be optimally designed and 
arranged to ensure, not only that the overall 
system's goals are met, but that the intended 
end-user has been optimally accommodated for. 
With sufficient testing and evaluation 
throughout the development cycle, then the 
selection of the components to use in the final 
telerobic system can promote efficient, 
error-free performance. It is recommended that 
whole-system simulation with full-scale 
mockups be used to help design the 
human-telerobot interface. It is contended that 
the use of simulation can facilitate this design 
and evaluation process. The use of simulation 
can also ensure that the hardware/software 
components have been selected to best 
accommodate the astronaut, instead of the 
astronaut having to make performance 
accommodations for the hardware/software 
components that have been selected. 


As was mentioned above, there are other 
advantages to simulating the 
human-teleoperator interface than simply 
serving as an aid in the selection and design of 
hardware/software components so that operator 
performance is optimized. Systems developers 
can also use the simulation system to test 
whether or not hardware components meet 
overall systems goals, and the simulation 
system can be used for subsequent training of 
the astronauts who will use the actual system. 
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